The systems and methods herein utilize machine vision techniques to track the locations and/or objects being viewed by an observer. Traditionally, gaze tracking algorithms have been considered as requiring two continuous data streams in order to produce tracking results: 1) head tracking methods to locate the position and orientation of the head within our three-dimensional world, and 2) eye tracking methods to detect glints produce by illumination sources along with the edges of pupils or other identifiable reference points on the surface of the eye to compute pivot angles and viewing directions of the eye relative to those glints. For accurate gaze tracking using an unobtrusive eyewear or headwear device, continually monitoring the position (relative to the surface of an eye) of the device itself including all cameras and illumination sources affixed to the device is an additional input to account for individual variations in head anatomy as well as small movements of the eyewear or headwear during use.
Applications that involve machine vision are becoming increasingly common-place. In part, this has arisen as a result of technological advances in the electronics and software development industries, and decreases in the cost of cameras, information processing units, and other electronic components. Gaze tracking, in particular, is increasingly being used in a number of diagnostic, human performance, and control applications. A small number of examples include monitoring the degree of fatigue of an individual, assessing driver or pilot awareness, assessing the effects of drugs or alcohol, diagnosing post-traumatic stress disorder, tracking human performance with age, determining the effectiveness of training or exercise on performance, assessing the effectiveness of television advertising or web-page designs by measuring ocular dwell times, magnifying or changing the brightness of specific objects or images (including words and sentences) under observation, controlling various aspects of games, acquiring foundational clinical data to assess neurological or cognitive disorders, diagnosing and monitoring degenerative eye conditions, and allowing individuals with limited or no mobility below the neck to communicate by controlling a computer cursor using one or more eyes and eyelids. Sectors and industries that utilize gaze tracking include military, medicine, security, human performance, gaming, sports medicine, rehabilitation engineering, police, research laboratories, and toys.
In almost all cases, an increase in the accuracy of gaze tracking leads to an increase in both the performance and ease-of-use of most applications. For example, with increased accuracy, ocular dwell times to quantify locations and fixation times on smaller objects or components of objects can be more accurately measured. Gaze tracking can be more effectively employed with portable devices that utilize smaller screens including mobile phones and hand-held displays. When gaze tracking is used to control a computer cursor involving the selection from a number of virtual objects or icons within a screen, an increased number of selectable objects can be displayed simultaneously because of the ability to choose smaller virtual objects or icons. An increased number of objects within each level of a selection process may increase the efficiency (i.e., reduced number of selection levels and/or reduced time) that a virtual object and associated action can be chosen.
With the advent of modern-day microelectronics and micro-optics, it is possible to unobtrusively mount the components for gaze tracking on eyewear (e.g., an eyeglass frame) or headwear (e.g., helmets, masks, goggles, virtual reality displays) including devices, such as those disclosed in U.S. Pat. Nos. 6,163,281, 6,542,081, or 7,488,294, 7,515,054, the entire disclosures of which are expressly incorporated by reference herein. Methods related to head tracking where one or more device-mounted scene cameras are used to tracking reference locations within our environment are disclosed in application Ser. No. 13/113,003, filed May 20, 2011, the entire disclosure of which is expressly incorporated by reference herein. Methods related to controlling the illumination of a scene based on camera images are disclosed in application Ser. No. 12/715,177, filed Mar. 1, 2010, the entire disclosure of which is expressly incorporated by reference herein. Methods related to measuring responses and reaction times of a device wearer are disclosed in application Ser. No. 13/113,006, filed May 20, 2011, the entire disclosure of which is expressly incorporated by reference herein; where improvements in spatial accuracy, as described in the present application, may contribute to improvements in the temporal accuracy of reaction time measurements.
Structures and reflections on the surface of the eye as well as the scene viewed by a device wearer may be imaged using high-precision micro-optics affixed to cameras within eyewear or headwear systems. The use of low-power, miniature cameras and electronics permits a full range of motion using a head-mounted system that is either tethered to other devices or non-tethered and (optionally) powered by a battery. Furthermore, recent advances in wireless telecommunications allow gaze tracking results to be transmitted in real-time to other computing, data storage, and/or control devices. As a result of these technological advances in a number of fields, an eyewear- or headwear-based gaze tracking system may be unobtrusive, light-weight, portable, and convenient to use with unrestricted mobility.